X-ray mammography remains a well-established technology, and x-ray tomosynthesis of the breast has been developed recently as discussed in the earlier-filed patent applications identified above. Clinical testing of tomosynthesis systems has been carried out, and the assignee of this patent specification, Hologic, Inc., has demonstrated at trade shows in this country a fused, multimode mammography/tomosynthesis system that takes either or both types of images, either in single or multiple compressions/immobilizations of the breast. Dedicated breast tomosynthesis systems also have been proposed.
Tomosynthesis as used in the systems and methods disclosed in this patent specification typically involves acquiring a plurality of tomosynthesis projection images Tp at respective angles relative to the breast, and using information describing at least these images Tp (or some of the Tp images) to reconstruct a plurality of tomosynthesis reconstructed images Tr representative of respective breast slices that have selective thicknesses and orientations and correspond to respective sections through or slices of the breast that typically are but need not be planar. For example, the slices can be curved in 3D space. In addition, conventional x-ray mammography images Mp can be acquired, in the same procedure that acquires the Tp images of a breast or in a different procedure and in the same or different compressions of the breast, and may be used in addition to the Tp images or in place of some of the Tp images, in reconstructing Tr images.
Mp images can be processed by various techniques that draw attention to selected portions or features of these images, such as CAD (computer aided detection) techniques that analyze the images to identify likely abnormalities and place markers on a breast image or representation that identify the location and in some cases the type or other information about the likely abnormalities. Some of the parent applications identified above describe applying CAD analysis to Tr and/or Tp images as well, or as an alternative or an adjunct to applying CAD to Mp images, and also describe how to improve the presentations of Tp and/or Tr images (collectively referred to here as T images), Mp images, and CAD and/or other information about the images to make the review by health professionals more effective and efficient.
CAD application to Mp images also is a well-established technology. See, for example, the mammography CAD products offered by R2 Technology, Inc. of Sunnyvale, Calif. and Patent Publications US 2002/0097902 and US 2004/0100476 and U.S. Pat. Nos. 5,970,164, and 6,630,937. In addition to the disclosure of applying CAD to Tp and Tr images in the related applications identified above, U.S. Pat. No. 6,748,044 discusses applying CAD to tomosynthesis images and U.S. Pat. No. 7,149,335 discusses detecting microcalcifications in tomosynthesis images and subtracting background to enhance visibility of the detected microcalcifications. Also possibly relevant to state-of-the-art may be US20030194121A1, U.S. Pat. No. 6,748,044, US20030215120A1, US20050002550A1, US20060269114A1, US20050089205A1, US20050113961A1, U.S. Pat. No. 7,142,633, US20060067473A1, US20060210131A1, US20070003117A1, US20070003118A1, US20070014448A1, US20070052700A1, US20070076928A1. The patents and publication identified in this paragraph also are incorporated by reference in this patent specification.
As more information about the breast becomes available from imaging, an important challenge is to present such information to the health professional effectively and efficiently such that screening for abnormalities can be done thoroughly and effectively, and yet in a reasonable time to be practical, and diagnostic assessment can also be facilitated. For a film mammographic study there usually are four films of the patient's breasts (at least for studies done in the U.S.A.), two CC views and 2 MLO views, that typically can be examined within a few minutes. However, with digital images and particularly tomosynthesis images Tp and Tr, there can be dozens of images and each can be viewed within different windows of pixel values centered at different levels of pixel values. Unless suitable display techniques are developed, information that is in the images may not be fully appreciated or utilized and/or the examination time may become so long as to be impractical, particularly for screening asymptomatic patients but also possibly for diagnostic or other purposes. This patent specification is directed to providing tomosynthesis and like images in a manner that would facilitate extracting useful information from them in a reasonable time.
In this patent specification, the notation Mp refers to a conventional mammogram, which is a two-dimensional x-ray projection image of a breast; the term Mp encompasses both a digital image as acquired by a flat panel detector or another imaging device and the image after conventional processing to prepare it for display to a health professional or for further processing or for storage, e.g. in the PACS system of a hospital or another institution. Mp also encompasses digitized film/screen mammograms. Tp refers to an image that is similarly two-dimensional but is taken at a respective tomosynthesis angle between the breast and the origin of the imaging X-rays (typically the focal spot of an X-ray tube), and also encompasses the image as acquired as well as the image after being processed for display or for some other use. Tr refers to an image that is reconstructed from images Tp, for example in the manner described in said earlier-filed patent applications, and represents a slice of the breast essentially as it would appear in a projection X-ray image of that slice at any desired angle, not only at an angle used for Tp or Mp images. In addition, a Tr image can represent a slice that conforms to any desired surface such as a flat or curved plane. Moreover, the process of reconstructing Tr images can use Mp images in addition to using Tp images or instead of one or more Tp images. The terms Tp, Tr and Mp also encompasses information, in whatever form, that is sufficient to describe such an image for display, further processing, or storage. The images Mp, Tp and Tr typically are in digital form before being displayed, and can be defined by information identifying properties of each pixel in a two-dimensional array of pixels, although other ways to describe the images can be used as well or instead. The pixel values typically relate to respective measured or estimated or computed responses to X-rays of corresponding volumes in the breast (voxels or columns of tissue). A Tr image can represents a thin slice of a breast, in which case it may consist of pixel values representing respective voxels (volume elements) of the breast that are in a single layer or a few layers transverse to the direction of the x-ray beam, or a Tr image may represent a thicker slice of the breast, in which case the pixel values of the thick-slice Tr image can represent columns of tissue along the direction of the x-ray beam and are calculated using known techniques such as, without limitation, a normalized projection of the pixels of several contiguous thin-slice images onto an image plane, a MW (maximum intensity projection), or some other way of combining the pixel values representing several thin-slice images. As a non-limiting example, a thin-slice Tr image can represent a 1 mm thick slice of the imaged breast and a thick-slice Tr image can represent a 5-20 mm thick slice of the breast. Thus, when a breast is compressed for x-ray imaging to a thickness of 5-6 cm, there can be 50-60 thin-slice Tr images and 3-12 thick-slice Tr images.
Microcalcifications seen in breast images are considered important clues in screening and/or diagnosis, and prior proposals have been directed to identifying particular specific patterns of microcalcifications or all microcalcifications, or at least those having specified characteristics such as size or density. This patent specification takes a different approach by not only necessarily seeking to identify and classify patterns of microcalcification distributions in images, or to identify or enhance all microcalcifications detectable in the image, but rather to facilitate the visualization of up to a certain number of selected suspected calcifications that meet various special thresholds in a given image or volume of tissue in ways that are particularly useful to the health professional. Calcifications often have a typical x-ray absorption characteristic, but not all objects with these absorption characteristics are calcifications or are of clinical value. Calcifications of clinical interest generally fall in a certain range of sizes and shapes and patterns. The largest calcifications are often benign. Linear ones also are often benign. Identifying all of the very smallest calcification-like objects runs the risk that some of them might represent noise and not true calcifications or reasonably suspected calcifications. One object of the approach disclosed in this patent specification is to reduce the number of identified possible or suspected calcifications which are of lower clinical value.
As one non-limiting example, the approach disclosed in this patent specification involves examining through computer processing the individual Tr images in a 3D set of such images to identify an initial set of possible calcifications that meets a threshold limiting the number of identified calcifications in a given Tr image, or in the entire 3D set or a selected subset of the entire 3D set, to a specified number of calcifications and/or number of pixels that are determined to correspond to calcifications. Preferably, the Tr images are presented for this examination after filtering with a mask that enhances high spatial frequency image features. The process removes from the initial set, pixels initially determined to relate to calcifications that are too large in area or too long in linear extent, and may additionally impose other constraints such as excluding initially determined calcifications that are not present in two adjacent Tr images, applying ligament removal and edge removal techniques, requiring at least a certain number of calcifications to be in a specified volume of the 3D set, and excluding calcifications that are in the initial and trailing Tr images in a stack of Tr images and thus are likely to be at the skin level and unlikely to have clinical significance. The removal processes are designed to remove calcifications that are likely to be devoid of clinical interest and to remove noise and other non-calcification objects. In displaying the results, one example is to show a scout view that generally conforms in shape to a projection of the breast and shows a distribution of identified calcifications as well as a current level that both (1) includes calcifications and (2) corresponds to a Tr image that is orthogonal to the scout image and is seen at a main image display. Alternatively, this information can be displayed using a ruler or other schematic display that does not conform in shape to a projection of the breast but still allows display of the current level and/or the locations of likely calcifications. Thus, the current level in the 2D scout or schematic view points to one or more corresponding Tr images. The scout or schematic view also shows a pointer to a next level that contains identified calcifications so that a user can conveniently click on that level and thus call on the main display the next Tr image of interest. The scout or schematic view may also indicate how many calcification clusters are in a given Tr image. In addition, the user can toggle the main display between showing a Tr image with or without highlighted calcifications therein. A health professional thus can quickly and effectively review Tr images that are likely to be of interest. The health professional can additionally select for the main display additional Tr images, Tr images that are for thick or thin slices and/or reconstructed in another orientation, Tp images and/or Mp images.
An alternative or additional process is to initially search Tp and/or Mp images rather than the 3D set of Tr images to identify likely calcifications of interest. This can be done using CAD as disclosed in the patent literature incorporated by reference in this patent specification, or by adapting the principles discussed above for identifying calcifications in Tr images and 3D volumes, or in some other manner. After likely calcifications of clinical significance are identified in the 2D Tp and/or Mp images, the 2D images can be displayed one at a time or several at a time, or only those 2D images that have identified calcifications can be displayed. When a user points to a likely calcification in a displayed 2D image, e.g., by a mouse click on the calcification in a 2D image, computer processing can search through the Tr images of the breast to identify and display the Tr image or Tr images that contain that calcification. This search through the Tr images can use knowledge about the location of the likely calcification in the 2D image, on parameters of the calcification such as its size, shape, pixel values and distribution of those values in the 2D image, and possibly other parameters. In case of an ambiguity, i.e., the calcification pointed to in the 2D image appears in two or more Tr images, all possibly relevant Tr images can be displayed to the user, singly or in a set or subsets or in cine mode, and the user may select for further use the ones that appear relevant or dismiss any that are not. This alternative can reduce processing time because the search for likely calcifications in the 2D images can be faster, and the subsequent search for the Tr slice(s) that contain a calcification to which the user pointed in a 2D image also can be faster. One reason for greater computational efficiency is that the 3D data set (the Tr images) need to be processed only after a likely calcification has been selected, so only a relatively small volume of the 3D set would need to be searched. Another advantage of this alternative may be if the search process is more sensitive and/or more accurate for 2D images. If so, then it is more likely to decrease the overall rate of false negatives and/or false positives in identifying likely calcifications, and thus improve the presentation of images to the health professional and make the overall process of assessing the images for screening, diagnosis, or other purposes, more effective and/or more efficient.
As another alternative or additional process, the user can point to any feature in one of the 2D images, not necessarily a likely calcification, even in 2D images that have not been processed to identify likely calcifications, and the process can attempt to identify and display for the user the Tr slice(s) in which that feature can be seen well, e.g., the feature is in best focus. The search through the 3D set of pixel data can use information about the feature to which the user pointed, such as location in the 2D image, size, shape, pixel values and distribution of those values, and possibly other parameters. If the feature is relatively unique and small, the process may correctly identify and display for the user one or only a few Tr slices. In other cases, such as in the case of a relatively large mass, or in the case of a feature in the 2D image that represents a superposition of two or more features in the breast that are along the x-ray beam path to that feature in the 2D image, there may be ambiguities as to which Tr image or images show the feature to which the user has pointed. Again, in such a case the process may similarly display the possibly pertinent Tr slices, or may display only those that meet a threshold of relevance, e.g., based on a calculation of their degree of relevance to the feature to which the user pointed.
The discussion above refers to highlighting calcifications. Other methods of emphasizing calcifications also are possible, and are in the scope of this patent specification. These include pointers such as arrows, circling or drawing lines, and the use of changing the color of the display of the calcification, or other methods of presenting visual cues to the viewer to look at specified areas on the image.+